Atsc over-the-air (ota) broadcast of public volumetric augmented reality (ar)

ABSTRACT

Techniques are described for using the Advanced Television Systems Committee (ATSC) 3.0 television protocol to deliver volumetric information for presentation on various displays using ATSC over-the-air communications channels.

FIELD

This application relates to technical advances necessarily rooted incomputer technology and directed to digital television, and moreparticularly to Advanced Television Systems Committee (ATSC) 3.0.

BACKGROUND

The Advanced Television Systems Committee (ATSC) 3.0 suite of standardsis a set of over a dozen industry technical standards as indicated inATSC A/300 for delivering the next generation of broadcast television.ATSC 3.0 supports delivery of a wide range of television servicesincluding, but not limited to, televised video, interactive services,non-real time delivery of data, and tailored advertising to a largenumber of receiving devices, from ultra-high-definition televisions towireless telephones. ATSC 3.0 also orchestrates coordination betweenbroadcast content (referred to as “over the air”) and related broadbanddelivered content and services (referred to as “over the top”). ATSC 3.0is designed to be flexible so that as technology evolves, advances canbe readily incorporated without requiring a complete overhaul of anyrelated technical standard. Present principles are directed to suchadvances as divulged below.

SUMMARY

As understood herein, broadcasting in ATSC 3.0 multicasts data from onesource to many receivers. ATSC 3.0 allows for User Defined tables in theLow Level Signaling starting point of Service Discovery (A/331 Standard)which are entirely user specific. Principles set forth in co-pendingU.S. patent application Ser. No. 16/952,581, incorporated herein byreference, may be used as appropriate in implementing presentprinciples.

As further understood herein, live AR content streaming over IP channelsspecific per user for their differing locations and perspectives,presents significant bandwidth and latency challenges, especially formobile AR devices and their existing data restrictions (caps, shared BWwith other services). Present techniques are provided to use existingATSC TV Broadcast infrastructure, existing Digital TV Channel data,which is replaced by a temporarily encoded stream of volumetric data toenable view-independent 3D data overlay (Volumetric AR).

The details of the present application, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an Advanced Television Systems Committee(ATSC) 3.0 system;

FIG. 2 is a block diagram showing components of the devices shown inFIG. 1 ;

FIG. 3 illustrates an example specific system for combining ATSCbroadcast volumetric audio-video (AV) content with AV content receivedfrom other than ATSC sources;

FIG. 3A illustrates a data structure;

FIG. 4 illustrates example ATSC broadcast transmitter logic in exampleflow chart format;

FIG. 5 illustrates example ATSC broadcast receiver logic in example flowchart format;

FIG. 6 illustrates a screen shot of an example AV presentation usingATSC volumetric content overlaid onto content received from a non-ATSCsource;

FIG. 7 illustrates a screen shot of an example AV presentation usingATSC alpha-numeric data overlaid onto content received from a non-ATSCsource;

FIG. 8 illustrates example logic in example flow chart format forgeo-tagging ATSC volumetric content;

FIGS. 9 and 10 illustrate screen shots of an example presentationconsistent with FIG. 8 ;

FIG. 11 illustrates example logic in example flow chart format foradding personal overlays onto ATSC broadcast volumetric content;

FIG. 12 illustrates an example screen shot consistent with FIG. 11 ;

FIG. 13 illustrates example logic in example flow chart format forallowing end users to modify ATSC broadcast volumetric content; and

FIG. 14 illustrates example logic in example flow chart format foraddressing latency issues.

DETAILED DESCRIPTION

This disclosure relates to technical advances in digital television suchas in Advanced Television Systems Committee (ATSC) 3.0 or highertelevision. An example system herein may include ATSC 3.0 sourcecomponents and client components, connected via broadcast and/or over anetwork such that data may be exchanged between the client and ATSC 3.0source components. The client components may include one or morecomputing devices including portable televisions (e.g., smart TVs,Internet-enabled TVs), portable computers such as laptops and tabletcomputers, and other mobile devices including smart phones andadditional examples discussed below. These client devices may operatewith a variety of operating environments. For example, some of theclient computers may employ, as examples, operating systems fromMicrosoft, or a Unix operating system, or operating systems produced byApple Computer or Google, such as Android®. These operating environmentsmay be used to execute one or more browsing programs, such as a browsermade by Microsoft or Google or Mozilla or other browser program that canaccess websites hosted by the Internet servers discussed below.

ATSC 3.0 source components may include broadcast transmission componentsand servers and/or gateways that may include one or more processorsexecuting instructions that configure the source components to broadcastdata and/or to transmit data over a network such as the Internet. Aclient component and/or a local ATSC 3.0 source component may beinstantiated by a game console such as a Sony PlayStation®, a personalcomputer, etc.

Information may be exchanged over a network between the clients andservers. To this end and for security, servers and/or clients caninclude firewalls, load balancers, temporary storages, and proxies, andother network infrastructure for reliability and security.

As used herein, instructions refer to computer-implemented steps forprocessing information in the system. Instructions can be implemented insoftware, firmware or hardware and include any type of programmed stepundertaken by components of the system.

A processor may be a general-purpose single- or multi-chip processorthat can execute logic by means of various lines such as address lines,data lines, and control lines and registers and shift registers.

Software modules described by way of the flow charts and user interfacesherein can include various sub-routines, procedures, etc. Withoutlimiting the disclosure, logic stated to be executed by a particularmodule can be redistributed to other software modules and/or combinedtogether in a single module and/or made available in a shareablelibrary. While flow chart format may be used, it is to be understoodthat software may be implemented as a state machine or other logicalmethod.

Present principles described herein can be implemented as hardware,software, firmware, or combinations thereof; hence, illustrativecomponents, blocks, modules, circuits, and steps are set forth in termsof their functionality.

Further to what has been alluded to above, logical blocks, modules, andcircuits can be implemented or performed with a processor, a digitalsignal processor (DSP), a field programmable gate array (FPGA) or otherprogrammable logic device such as an application specific integratedcircuit (ASIC), discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A processor can be implemented by a controller orstate machine or a combination of computing devices.

The functions and methods described below, when implemented in software,can be written in an appropriate language such as but not limited tohypertext markup language (HTML)-5, Java/JavaScript, C# or C++, and canbe stored on or transmitted through a computer-readable storage mediumsuch as a random access memory (RAM), read-only memory (ROM),electrically erasable programmable read-only memory (EEPROM), compactdisk read-only memory (CD-ROM) or other optical disk storage such asdigital versatile disc (DVD), magnetic disk storage or other magneticstorage devices including removable thumb drives, etc. A connection mayestablish a computer-readable medium. Such connections can include, asexamples, hard-wired cables including fiber optics and coaxial wires anddigital subscriber line (DSL) and twisted pair wires.

Components included in one embodiment can be used in other embodimentsin any appropriate combination. For example, any of the variouscomponents described herein and/or depicted in the Figures may becombined, interchanged, or excluded from other embodiments.

“A system having at least one of A, B, and C” (likewise “a system havingat least one of A, B, or C” and “a system having at least one of A, B,C”) includes systems that have A alone, B alone, C alone, A and Btogether, A and C together, B and C together, and/or A, B, and Ctogether, etc.

Turning to FIG. 1 , an example of an ATSC 3.0 source component islabeled “broadcaster equipment” 10 and may include over-the-air (OTA)equipment 12 for wirelessly broadcasting, typically via orthogonalfrequency division multiplexing (OFDM) in a one-to-many relationship,television data to plural receivers 14 such as ATSC 3.0 televisions. Oneor more receivers 14 may communicate with one or more companion devices16 such as remote controls, tablet computers, mobile telephones, and thelike over a short range, typically wireless link 18 that may beimplemented by Bluetooth®, low energy Bluetooth, other near fieldcommunication (NFC) protocol, infrared (IR), etc.

Also, one or more of the receivers 14 may communicate, via a wiredand/or wireless network link 20 such as the Internet, with over-the-top(OTT) equipment 22 of the broadcaster equipment 10 typically in aone-to-one relationship. The OTA equipment 12 may be co-located with theOTT equipment 22 or the two sides 12, 22 of the broadcaster equipment 10may be remote from each other and may communicate with each otherthrough appropriate means. In any case, a receiver 14 may receive ATSC3.0 television signals OTA over a tuned-to ATSC 3.0 television channeland may also receive related content, including television, OTT(broadband). Note that computerized devices described in all of thefigures herein may include some or all of the components set forth forvarious devices in FIGS. 1 and 2 .

Referring now to FIG. 2 , details of examples of components shown inFIG. 1 may be seen. FIG. 2 illustrates an example protocol stack thatmay be implemented by a combination of hardware and software. Using theATSC 3.0 protocol stack shown in FIG. 2 and modified as appropriate forthe broadcaster side, broadcasters can send hybrid service delivery inwhich one or more program elements are delivered via a computer network(referred to herein as “broadband” and “over-the-top” (OTT)) as well asvia a wireless broadcast (referred to herein as “broadcast” and“over-the-air” (OTA)). FIG. 2 also illustrates an example stack withhardware that may be embodied by a receiver.

Disclosing FIG. 2 in terms of broadcaster equipment 10, one or moreprocessors 200 accessing one or more computer storage media 202 such asany memories or storages described herein may be implemented to provideone or more software applications in a top-level application layer 204.The application layer 204 can include one or more software applicationswritten in, e.g., HTML5/JavaScript running in a runtime environment.Without limitation, the applications in the application stack 204 mayinclude linear TV applications, interactive service applications,companion screen applications, personalization applications, emergencyalert applications, and usage reporting applications. The applicationstypically are embodied in software that represents the elements that theviewer experiences, including video coding, audio coding and therun-time environment. As an example, an application may be provided thatenables a user to control dialog, use alternate audio tracks, controlaudio parameters such as normalization and dynamic range, and so on.

Below the application layer 204 is a presentation layer 206. Thepresentation layer 206 includes, on the broadcast (OTA) side, broadcastaudio-video playback devices referred to as Media Processing Units (MPU)208 that, when implemented in a receiver, decode and playback, on one ormore displays and speakers, wirelessly broadcast audio video content.The MPU 208 is configured to present International Organization forStandardization (ISO) base media file format (BMFF) data representations210 and video in high efficiency video coding (HEVC) with audio in,e.g., Dolby audio compression (AC)-4 format. ISO BMF is a general filestructure for time-based media files broken into “segments” andpresentation metadata. Each of the files is essentially a collection ofnested objects each with a type and a length. To facilitate decryption,the MPU 208 may access a broadcast side encrypted media extension(EME)/common encryption (CENC) module 212.

FIG. 2 further illustrates that on the broadcast side the presentationlayer 206 may include signaling modules, including either motionpictures expert group (MPEG) media transport protocol (MMTP) signalingmodule 214 or real-time object delivery over unidirectional transport(ROUTE) signaling module 216 for delivering non-real time (NRT) content218 that is accessible to the application layer 204. NRT content mayinclude but is not limited to stored replacement advertisements. Audiovideo (AV) streams are contained in ROUTE sessions. Layered codingtransport (LCT) channels are setup within a ROUTE session. Each LCTchannel carries either video or audio or captions or other data.

On the broadband (OTT or computer network) side, when implemented by areceiver the presentation layer 206 can include one or more dynamicadaptive streaming over hypertext transfer protocol (HTTP) (DASH)player/decoders 220 for decoding and playing audio-video content fromthe Internet. To this end the DASH player 220 may access a broadbandside EME/CENC module 222. The DASH content may be provided as DASHsegments 224 in ISO/BMFF format.

As was the case for the broadcast side, the broadband side of thepresentation layer 206 may include NRT content in files 226 and may alsoinclude signaling objects 228 for providing play back signaling.

Below the presentation layer 206 in the protocol stack is a sessionlayer 230. The session layer 230 includes, on the broadcast side, eitherMMTP protocol 232 or ROUTE protocol 234.

On the broadband side the session layer 230 includes HTTP protocol 236which may be implemented as HTTP-secure (HTTP(S)). The broadcast side ofthe session layer 230 also may employ a HTTP proxy module 238 and aservice list table (SLT) 240. The SLT 240 includes a table of signalinginformation which is used to build a basic service listing and providebootstrap discovery of the broadcast content. Media presentationdescriptions (MPD) are included in the “ROUTE Signaling” tablesdelivered over user datagram protocol (UDP) by the ROUTE transportprotocol.

A transport layer 242 is below the session layer 230 in the protocolstack for establishing low-latency and loss-tolerating connections. Onthe broadcast side the transport layer 242 uses user datagram protocol(UDP) 244 and on the broadband side transmission control protocol (TCP)246.

The example non-limiting protocol stack shown in FIG. 2 also includes anetwork layer 248 below the transport layer 242. The network layer 248uses Internet protocol (IP) on both sides for IP packet communication,with multicast delivery being typical on the broadcast side and unicastbeing typical on the broadband side.

Below the network layer 248 is the physical layer 250 which includesbroadcast transmission/receive equipment 252 and computer networkinterface(s) 254 for communicating on the respective physical mediaassociated with the two sides. The physical layer 250 converts InternetProtocol (IP) packets to be suitable to be transported over the relevantmedium and may add forward error correction functionality to enableerror correction at the receiver as well as contain modulation anddemodulation modules to incorporate modulation and demodulationfunctionalities. This converts bits into symbols for long distancetransmission as well as to increase bandwidth efficiency. On the OTAside the physical layer 250 typically includes a wireless broadcasttransmitter to broadcast data wirelessly using orthogonal frequencydivision multiplexing (OFDM) while on the OTT side the physical layer250 includes computer transmission components to send data over theInternet.

A DASH Industry Forum (DASH-IF) profile formatted data sent through thevarious protocols (HTTP/TCP/IP) in the protocol stack may be used on thebroadband side. Media files in the DASH-IF profile formatted data basedon the ISO BMFF may be used as the delivery, media encapsulation andsynchronization format for both broadcast and broadband delivery.

Each receiver 14 typically includes a protocol stack that iscomplementary to that of the broadcaster equipment.

A receiver 14 in FIG. 1 may include, as shown in FIG. 2 , anInternet-enabled TV with an ATSC 3.0 TV tuner (equivalently, set top boxcontrolling a TV) 256. The receiver 14 may be an Android®-based system.The receiver 14 alternatively may be implemented by a computerizedInternet enabled (“smart”) telephone, a tablet computer, a computer gameconsole or controller or head-mounted display (HMD), a notebookcomputer, a wearable computerized device, and so on. Regardless, it isto be understood that the receiver 14 and/or other computers describedherein is configured to undertake present principles (e.g., communicatewith other devices to undertake present principles, execute the logicdescribed herein, and perform any other functions and/or operationsdescribed herein).

Accordingly, to undertake such principles the receiver 14 can beestablished by some or all of the components shown in FIG. 1 . Forexample, the receiver 14 can include one or more displays 258 that maybe implemented by a high definition or ultra-high definition “4K” orhigher flat screen and that may or may not be touch-enabled forreceiving user input signals via touches on the display. The receiver 14may also include one or more speakers 260 for outputting audio inaccordance with present principles, and at least one additional inputdevice 262 such as, e.g., an audio receiver/microphone for, e.g.,entering audible commands to the receiver 14 to control the receiver 14.The example receiver 14 may further include one or more networkinterfaces 264 for communication over at least one network such as theInternet, a WAN, a LAN, a PAN etc. under control of one or moreprocessors 266. Thus, the interface 264 may be, without limitation, aWi-Fi transceiver, which is an example of a wireless computer networkinterface, such as but not limited to a mesh network transceiver. Theinterface 264 may be, without limitation, a Bluetooth® transceiver,Zigbee® transceiver, Infrared Data Association (IrDA) transceiver,Wireless USB transceiver, wired USB, wired LAN, Powerline or Multimediaover Coax Alliance (MoCA). It is to be understood that the processor 266controls the receiver 14 to undertake present principles, including theother elements of the receiver 14 described herein such as, forinstance, controlling the display 258 to present images thereon andreceiving input therefrom. Furthermore, note the network interface 264may be, e.g., a wired or wireless modem or router, or other appropriateinterface such as, e.g., a wireless telephony transceiver, or Wi-Fitransceiver as mentioned above, etc.

In addition to the foregoing, the receiver 14 may also include one ormore input ports 268 such as a high-definition multimedia interface(HDMI) port or a USB port to physically connect (using a wiredconnection) to another CE device and/or a headphone port to connectheadphones to the receiver 14 for presentation of audio from thereceiver 14 to a user through the headphones. For example, the inputport 268 may be connected via wire or wirelessly to a cable or satellitesource of audio video content. Thus, the source may be a separate orintegrated set top box, or a satellite receiver. Or the source may be agame console or disk player.

The receiver 14 may further include one or more computer memories 270such as disk-based or solid-state storage that are not transitorysignals, in some cases embodied in the chassis of the receiver asstandalone devices or as a personal video recording device (PVR) orvideo disk player either internal or external to the chassis of thereceiver for playing back audio video (AV) programs or as removablememory media. Also, in some embodiments, the receiver 14 can include aposition or location receiver 272 such as but not limited to a cellphonereceiver, global positioning satellite (GPS) receiver, and/or altimeterthat is configured to e.g., receive geographic position information fromat least one satellite or cellphone tower and provide the information tothe processor 266 and/or determine an altitude at which the receiver 14is disposed in conjunction with the processor 266. However, it is to beunderstood that that another suitable position receiver other than acellphone receiver, GPS receiver and/or altimeter may be used inaccordance with present principles to determine the location of thereceiver 14 in e.g., all three dimensions.

Continuing the description of the receiver 14, in some embodiments thereceiver 14 may include one or more cameras 274 that may include one ormore of a thermal imaging camera, a digital camera such as a webcam,and/or a camera integrated into the receiver 14 and controllable by theprocessor 266 to gather pictures/images and/or video in accordance withpresent principles. Also included on the receiver 14 may be a Bluetooth®transceiver 276 or other Near Field Communication (NFC) element forcommunication with other devices using Bluetooth® and/or NFC technology,respectively. An example NFC element can be a radio frequencyidentification (RFID) element.

Further still, the receiver 14 may include one or more auxiliary sensors278 (such as a motion sensor such as an accelerometer, gyroscope,cyclometer, or a magnetic sensor and combinations thereof), an infrared(IR) sensor for receiving IR commands from a remote control, an opticalsensor, a speed and/or cadence sensor, a gesture sensor (for sensinggesture commands) and so on providing input to the processor 266. An IRsensor 280 may be provided to receive commands from a wireless remotecontrol. A battery (not shown) may be provided for powering the receiver14.

The companion device 16 may incorporate some or all of the elementsshown in relation to the receiver 14 described above.

FIG. 3 illustrates an example system consistent with present principles.A source 300 such as a computer game server or console of augmentedreality (AR) volumetric data including real time (live) changes to thevolumetric data sends the data to a temporal codec 302 to be compressed.Along with the volumetric data, side-band data for providing an initialseeding for the volumetric content represented by the data is sent. Thecompressed data is sent from the codec 302 to an ATSC broadcasterassembly 304, such as an ATSC 3.0 or higher broadcaster assembly.

Volumetric refers to data within a specific volume that isview-independent but includes sufficient information that allows aprocess to extract explicit views. Some of that information may include,e.g., GPS coordinates of objects within the volumetric data.

An example of volumetric data may be a computer simulation such as acomputer game with “blocky” 3D terrain that players explore. An exampleof such a game is “Minecraft”. A 3D volume of width×height×depth blocksor cells is used to store information about the “Minecraft world”. Eachcell may contain information describing a 3D block, each a piece ofground, dirt, rock, tree, etc. Empty space may be encoded in a moreefficient manner.

This same basic approach can be applied to 3D data that can be used forAugmented Reality (AR), in which 3D data is overlaid onto a real-worldlocation. The “blocky” world that makes up the AR is encoded into thisvolumetric data. However, any 3D object or informational representationcan be encoded as volumetric data.

The benefit of encoding AR objects as volumetric data is that they onlyneed to be encoded once if they are static objects (for instance, an ARroad sign) and temporarily updated if they are dynamic (as an example,wheels turning on a car). For a virtual world to be overlaid as AR, mostof the world typically is static (ground, hills, mountains, trees,buildings, etc.) with periodic changes (clouds moving, characters andanimals moving around, trees animating, etc.) The encoding of volumetricdata is not dependent on how the data is viewed, i.e., it isview-independent, compared to other techniques to encode 3D data such asgeometry, textures, materials, that also require rendering for aspecific viewpoint (and hence are view-dependent). To render volumetricdata for a specific view, various methods can be deployed, including butlimited to ray tracing or ray casting into the dataset.

It is the view-independence of 3D volumetric data that enables broadcastdelivery mechanism, in which everyone receives the same data. Only theprocess of how the data is interpreted, which is the rendering-specificview from the 3D volumetric data, needs to be done differently for eachclient/user of the data.

ATSC 3.0 can transmit, in addition to digital audio and video data,arbitrary data as well. Currently, just using the extra data channels inATSC 3.0, ten mbit/second of 3D AR data can be sent. If TV channels wereremoved, twenty-five mbit/s or more may be made available.Photorealistic 3D volumetric data can be compressed in as little as 5MB-LOMB. Based on using ATSC 3.0 data channel, about one MB can obtainedin one second. So, a full volumetric model (character, scene, etc.) canstreamed OTA in about five to ten seconds at most.

The bandwidth of OTA ATSC 3.0 data channel would not be enough forreal-time interactive frame rates of thirty frames per second, but asmentioned, most content is static and only small changes happen at every1/30th of a second.

For the 10 mbit OTA example, 333 Kbit/s can be available frame to framechanges of any volumetric data. Present techniques accordingly us threephases to the OTA streaming of volumetric data.

In the first, or seed, phase, before a volumetric model data can be usedby a client (receiver) to show an appropriate view of 3D model, enoughdata is needed to provide a based dataset before changes can be applied.“Seed” data accordingly is sent first. The seed data contains all thestatic elements of the volumetric data. For some 3D models orapplications, for instance the Minecraft example, this seed data couldbe a significant portion of the volumetric data.

A portion of this data can be streamed constantly within the OTA datastream, for example, one-half or five mbit/s in this example. The seeddata is broken into numbered portions and streamed constantly. Once aclient has received all numbered portions, the client can proceed to thenext phase.

The next phase may be termed the “Full Frame (FF) Phase” in which aclient device looks at the temporal data (for instance, the other 50% ofthe example above, again five mbit/s) sent OTA. A “frame” of changes tothe static seed data is broken up into pieces and broadcast in thetemporal data section.

Once a client receives all the pieces of the frame data (they come in 5MB chunks), the client receiver can now show the volumetric content.

In the third phase, which may be termed “Frame-to-Frame (F2F) Phase”,periodically after the frame data, frame-to-frame data is broadcast.This data just contains the small changes from one frame of changingcontent to the next, e.g., 1/30th of a frame of animation to the next.

An example of the flow of data in the OTA broadcast according to theabove may be:

[seed data chunk 50%]/[full frame data chunk or frame-to-frame chunk50%]

[SEED 1]/[FF1-1]->[SEED 2]/[FF1-2]->[SEED 1]/[F2F1]->[SEED2]/[F2F2]->[SEED 1]/[F2F4]->[SEED 2]/[FF2-1]->[SEED 1]/[FF2-2]-> etc . .. .

Returning to FIG. 3 , the data is broadcast from the assembly 304over-the-air (OTA) to receiver assemblies 306 (only a single receiverassembly 306 shown in FIG. 3 for clarity.)

In the example shown, the receiver assembly 306 includes an ATSC 3.0 orhigher receiver 308 (which typically includes an antenna) and a decoder310 that decodes information from the receiver 308. The decodedvolumetric data is presented on a display 312 under control of theprocessor of the receiver assembly 306, which may include appropriatecomponents from FIG. 2 in addition to those shown n FIG. 3 .

The receiver assembly 306 also includes a non-ATSC interface 316 such asa Wi-Fi transceiver, Bluetooth transceiver, 5G or other wirelesstelephony transceiver, Ethernet interface, etc. that communicates with asource 314 of overlay information to be presented on the display 312along with the volumetric content received via ATSC OTA broadcast.

Accordingly, client receiver assemblies with ATSC receivers can “tunein” to the AR broadcast and after a few seconds of decoding theside-band data, can visualize live AR overlay of 3D visuals with audio,from their device's location and perspective.

FIG. 4 illustrates further. Commencing at block 400, 3D volumetric datais received. The data and/or changes thereto in real time is compressedat block 402. If desired, the data to be broadcast can be tagged asgeo-located to a very specific location on the globe, down to thenearest cm or mm. This can be used for AR broadcasts of publicinformation such as public buildings and sites, public events, etc.Block 404 indicates that the compressed 3D volumetric data with changesand side-band data is broadcast OTA via ATSC (e.g., ATSC 3.0).

FIG. 5 illustrates complementary receiver side logic. The ATSC broadcastdata is received at block 500 and the side-band data decoded to startdecoding of the 3D volumetric data at block 502. Proceeding to block504, using the side-band data, presentation of the 3D volumetric data isseeded. The 3D volumetric data is presented at block 506 along withaudio-video information received from the non-ATSC interface 316 shownin FIG. 3 .

FIG. 6 illustrates that for instance, a display 600 such as any displayherein may present an image of a museum 602 holding a special event onits dinosaur bone collection that uses a promotional broadcast feed froma local ATSC broadcaster to provide animated dinosaurs 606 in AR on thefront steps 604 of the museum along with other event information.

FIG. 7 illustrates alternate use cases in which a display 700 such asany display herein presents alpha-numeric information 704 such as AR busschedules and routes are overlaid onto an image 702 of a geographic sitesuch as a city. Other examples of the alpha-numeric AR overlay includeemergency/disaster planning information guiding AR users to safe areas,AR game stats overlaid at sports and eSports venues, etc. usinglocalized ATSC transmitters located at the Stadiums.

FIG. 8 illustrates. Commencing at block 800, geo tags are identified inthe ATSC broadcast data so that at block 802 the accompanying AR contentcan be overlaid onto images of the desired geographic location specifiedby the geo tags.

Or, as shown in FIG. 9 the AR data may not include geo tags (orequivalently may be tagged as not being tied to any particularlocation), giving clients the option to anchor the AR content whereverthey want. In the example of FIG. 9 , an AR version of a computer gamecan be received via ATSC for presentation on a display 900 such as anydisplay herein, with a prompt 902 for the user to select which object ina real-world field of view (e.g., as seen through an AR HMD) the userwishes the AR version of the computer game to be overlaid on. In theexample shown, two tables 904 are in the real-world field of view, andas shown in FIG. 10 the user has selected “table 1” on which the ARvolumetric data (in this example, a computer game consisting ofcharacters 1002) are overlaid. The real-world images may be receivedfrom the non-ATSC interface 3416 shown in FIG. 3 .

Thus, the computer game is scaled and “placed” on table 1 as avolumetric 3D chunk. Users can walk around the table to see the gameaction from different perspectives using their ATSC capable Smartphones,Tablets or AR Glasses, etc.

The volumetric data is encoded in such a way as to be compatible withexisting ATSC (e.g., 3.0) encoding standards and allow for efficient,high equality decoding on client devices equipped with ATSC receivers.The content broadcast over the air is therefore completely free to allpeople with ATSC receivers and the data bandwidth does not conflict withtheir existing data delivery services (Wi-Fi, 5G, etc.). In addition toOTA Public AR, private or group AR overlays can be added on top of theATSC derived data using IP traffic over other data delivery services(Wi-Fi, 5G, etc.).

The benefit of this hybrid ATSC data plus IP data model is that thebandwidth is shared between different radio frequencies and IP trafficcan be kept to a minimum, only for the private data that is required.

FIG. 11 illustrates an additional example in which, referring back tothe computer game of FIGS. 9 and 10 , users can at block 1100 log into agame service network over an IP channel and add additional ARinformation at block 1102 on top of the public feed that everyone elseis viewing for presentation of the augmented hybrid content at block1104.

As illustrated in related FIG. 12 , a display 1200 such as any displayherein can present the ATSC OTA information 1202 overlaid with theprivate overlay information 1204, which in the example shown may includemessage chats (that may be age restricted) or selected playerstatistics, biographies, etc.

It should be noted as indicated at block 1300 in FIG. 13 that the OTAbroadcast of AR is one-way, delivering consistent live AR content toview, listen and feel (via client haptics). Alterations to the broadcastvolumetric data may be received at block 1302 responsive to an end useremploying, e.g., a computer game controller, with the alterations beingreceived via standard IP channels (Ethernet, Wi-Fi, 5G, etc.). Forexample, for a live volumetric AR computer game competition, players usetheir client devices to send player inputs (virtual character actions)to a central server at block 1302 using standard IP channels or via adirect IP (5G, etc.) connected game controller. At block 1304 the serveridentifies the many player inputs and computes the changes to thecomputer game competition volumetric world. Proceeding to block 1306,the server sends the changes to volumetric data to the ATSC broadcasterassembly 304 shown in FIG. 3 . This may be done using a high-speed fiberconnection (Internet Backbone). The modified AR volumetric data isbroadcast OTA at block 1308 as described previously.

FIG. 14 illustrates that to account for latencies in the systemsdescribed above, at block 1400 the volumetric AR data can be receivedencoded with timing data and future predicted changes. The volumetric ARis broadcast and proceeding to block 1402, a receiving client candetermine from the time stamps compared to their local clock how much ofthe predicted data they must use. Alternatively, at block 1404 theclient devices themselves generate the predicted volumetric changesbased on the timing data.

Present techniques may be used for every day, low-impact, highlyreliable broadcasts of games and virtual events in AR for spectating.Players interacting with games on this service could be doing this inthe current form, with direct IP communication.

The methods described herein may be implemented as software instructionsexecuted by a processor, suitably configured application specificintegrated circuits (ASIC) or field programmable gate array (FPGA)modules, or other manner as would be appreciated by those skilled inthose art. Where employed, the software instructions may be embodied ina non-transitory device such as a CD ROM or Flash drive. The softwarecode instructions may alternatively be embodied in a transitoryarrangement such as a radio or optical signal, or via a download overthe Internet.

It will be appreciated that whilst present principals have beendescribed with reference to some example embodiments, these are notintended to be limiting, and that various alternative arrangements maybe used to implement the subject matter claimed herein.

What is claimed is:
 1. A digital television transmitter assemblycomprising: at least one processor programmed with instructions to:send, via over-the-air (OTA) transmission, to at least one displayaugmented reality (AR) information to be presented on the display, theAR information comprising view-independent volumetric data.
 2. Thedigital television transmitter assembly of claim 1, wherein the digitaltelevision comprises advanced television systems committee (ATSC) 3.0television.
 3. The digital television transmitter assembly of claim 1,wherein the AR information comprises computer game information.
 4. Thedigital television transmitter assembly of claim 1, wherein the ARinformation comprises alpha numeric information.
 5. The digitaltelevision transmitter assembly of claim 1, wherein the AR informationcomprises geographic location information.
 6. The digital televisiontransmitter assembly of claim 1, comprising at least one serverprogrammed with instructions to: receive input from plural receivers ofthe AR information; alter AR information to be broadcast according tothe input; and provide altered AR information to the digital televisiontransmitter assembly for transmission thereof.
 7. A digital televisionreceiver assembly comprising: at least one display for presentingaugmented reality (AR) images received from a digital televisionbroadcast; and at least one processor configured with instructions for:controlling the display to present the AR images; and present on thedisplay, along with the AR images, images from a non-Advanced TelevisionSystems Committee (ATSC) source.
 8. The digital television receiverassembly of claim 7, wherein the non-ATSC source comprises a Wi-Fisource.
 9. The digital television receiver assembly of claim 7, whereinthe non-ATSC source comprises a wireless telephony source.
 10. Thedigital television receiver assembly of claim 7, wherein the non-ATSCsource comprises an Ethernet source.
 11. The digital television receiverassembly of claim 7, wherein the processor is configured withinstructions for: presenting the AR images in accordance with side-bandinformation received in the digital television broadcast.
 12. Thedigital television receiver assembly of claim 7, wherein the processoris configured with instructions for: identifying the images from thenon-ATSC source based on user input; and overlaying the AR images ontothe images from the non-ATSC source according to the user input.
 13. Asystem comprising: at least one Advanced Television Systems Committee(ATSC) over-the-air broadcast assembly configured to broadcast augmentedreality (AR) volumetric content; at least one ATSC receiver assemblycomprising at least one ATSC receiver configured to receive the ARvolumetric content from the broadcast assembly and at least one non-ATSCcommunication interface configured to receive non-ATSC content andpresent in one image on at least one display both the AR volumetriccontent from the broadcast assembly and the non-ATSC content.
 14. Thesystem of claim 13, wherein the AR volumetric content comprises computergame content.
 15. The system of claim 13, wherein the AR volumetriccontent comprises alpha numeric information.
 16. The system of claim 13,comprising at least one server programmed with instructions to: receiveinput from plural receivers of the AR volumetric content; alter ARvolumetric content to be broadcast according to the input; and providealtered AR volumetric content to the ATSC broadcast assembly fortransmission thereof.
 17. The system of claim 13, wherein the non-ATSCcommunication interface comprises a Wi-Fi interface.
 18. The system ofclaim 13, wherein the non-ATSC communication interface comprises anEthernet interface.
 19. The system of claim 13, wherein the non-ATSCcommunication interface comprises a wireless telephony interface. 20.The system of claim 13, wherein the ATSC receiver assembly is configuredfor: identifying selection of at least one image from the non-ATSCcommunication interface based on user input; and overlaying the ARvolumetric content from the ATSC broadcast assembly onto the images fromthe non-ATSC communication interface according to the user input.